feat: add UFC scraper + FMP earnings API + tests (#2) #9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* docs: add CLAUDE.md for Claude Code context Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * chore: pin dependency versions and add dev requirements Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * test: add pytest suite with 143 tests at 68% coverage Tests cover utils (make_slug, generate_ics), base classes (Event, CalendarBase, IntegrationBase), and integrations (investing, imdb, shows, wwe, weather, sportsdb, weather_geocode) with both pure function tests and mocked HTTP integration tests. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * ci: add GitHub Actions test workflow with coverage Runs pytest with coverage on every push and PRs to main. Uses Python 3.12 (stable) with pip caching. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * test: add tests for 0% coverage integrations (moviedb, releases, thetvdb, twitch, google_sheets, routes) 50 new tests covering all previously untested integration modules. Coverage improved from 68% to 87% (143 → 193 tests). Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * test: comprehensive shows.py tests covering scraping, tmsid, episodes, and fetch_events Shows.py coverage: 21% → 99% (22 new tests). Total suite: 215 tests, 92% coverage, 0.51s. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * docs: add scraper consolidation design plan Documents the plan to retire sync2cal-custom-scraper and migrate 7,959 category SOURCE URLs to events-api. Includes URL rewrite rules for all 9 integrations and rollback plan. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * docs: add /tv/genre/ URL pattern to migration design doc Discovered 11 categories use /tv/genre/{slug} in addition to /tv/platform/{slug}. Updated URL mapping table and rewrite rules. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * feat: auto-refresh TheTVDB bearer token on expiry Instead of requiring a manually-set bearer token env var, the integration now auto-logins with the API key and caches the token in memory. On 401 responses, it invalidates the cache, re-logins, and retries once. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * refactor: remove THE_TVDB_BEARER_TOKEN env var dependency Auto-login with API key is sufficient — no need for a separate bearer token env var. Updated tests to mock _login_to_thetvdb instead of setting the env var. Added test for 401 retry flow. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * feat: support Google Sheets service account from env var Add GOOGLE_SHEETS_SERVICE_ACCOUNT_JSON env var support so credentials can be stored as an environment variable in Railway instead of requiring a file on disk. Falls back to file-based auth if env var is not set. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * fix: use full browser User-Agent for investing.com requests investing.com rejects requests with truncated User-Agent "Mozilla/5.0", returning 403 Forbidden. Using a complete Chrome UA string fixes this. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * feat: replace investing.com earnings scraping with FMP API investing.com blocks datacenter IPs with 403 responses, making earnings scraping unreliable from Railway. Replace with Financial Modeling Prep (FMP) API which provides structured JSON earnings calendar data. - Earnings now fetched from FMP /stable/earnings-calendar endpoint - IPO scraping kept on investing.com (FMP IPO requires paid plan) - Added _format_revenue() helper for human-readable revenue display - Requires FMP_API_KEY environment variable - Updated tests: replaced parse_earnings/build_earnings_payload tests with FMP-specific tests (fetch_earnings_fmp, _format_revenue, etc.) Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> * feat: add UFC scraper that fetches events directly from UFC.com - Scrapes UFC.com events pages for accurate event data - Uses data-timestamp attribute for precise event times - Extracts fighter names, venues, and fight card info - Deduplicates events by URL slug and UID - New endpoint: /ufc/events This replaces TheSportsDB for UFC events which had dates off by ~1 day. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Happy <yesreply@happy.engineering>
|
You have run out of free Bugbot PR reviews for this billing cycle. This will reset on March 6. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
Contributor
Author
|
Closing: UFC scraper was already merged via PR #10. This PR from main branch is stale. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Generated with Claude Code via Happy
Generated with Claude Code via Happy
Tests cover utils (make_slug, generate_ics), base classes (Event, CalendarBase, IntegrationBase), and integrations (investing, imdb, shows, wwe, weather, sportsdb, weather_geocode) with both pure function tests and mocked HTTP integration tests.
Generated with Claude Code via Happy
Runs pytest with coverage on every push and PRs to main. Uses Python 3.12 (stable) with pip caching.
Generated with Claude Code via Happy
50 new tests covering all previously untested integration modules. Coverage improved from 68% to 87% (143 → 193 tests).
Generated with Claude Code via Happy
Shows.py coverage: 21% → 99% (22 new tests).
Total suite: 215 tests, 92% coverage, 0.51s.
Generated with Claude Code via Happy
Documents the plan to retire sync2cal-custom-scraper and migrate 7,959 category SOURCE URLs to events-api. Includes URL rewrite rules for all 9 integrations and rollback plan.
Generated with Claude Code via Happy
Discovered 11 categories use /tv/genre/{slug} in addition to /tv/platform/{slug}. Updated URL mapping table and rewrite rules.
Generated with Claude Code via Happy
Instead of requiring a manually-set bearer token env var, the integration now auto-logins with the API key and caches the token in memory. On 401 responses, it invalidates the cache, re-logins, and retries once.
Generated with Claude Code via Happy
Auto-login with API key is sufficient — no need for a separate bearer token env var. Updated tests to mock _login_to_thetvdb instead of setting the env var. Added test for 401 retry flow.
Generated with Claude Code via Happy
Add GOOGLE_SHEETS_SERVICE_ACCOUNT_JSON env var support so credentials can be stored as an environment variable in Railway instead of requiring a file on disk. Falls back to file-based auth if env var is not set.
Generated with Claude Code via Happy
investing.com rejects requests with truncated User-Agent "Mozilla/5.0", returning 403 Forbidden. Using a complete Chrome UA string fixes this.
Generated with Claude Code via Happy
investing.com blocks datacenter IPs with 403 responses, making earnings scraping unreliable from Railway. Replace with Financial Modeling Prep (FMP) API which provides structured JSON earnings calendar data.
Generated with Claude Code via Happy
This replaces TheSportsDB for UFC events which had dates off by ~1 day.
Generated with Claude Code via Happy